Learning to Calibrate - Estimating the Hand-eye Transformation Without Calibration Objects

Hand-eye calibration is a method to determine the transformation linking between robot and camera coordinate systems. Conventional algorithms use grid poses, corresponding both of which are used in main procedure. Although such methods yield good accuracy suitable for offline applications, they not applicable dynamic environment as robotic-assisted minimally invasive surgery (RMIS) because changes setup can be disruptive time-consuming workflow it requires yet another In this paper, we propose neural network-based hand-eye that does require poses from but only uses motion surgical instruments frame their input recover matrix. The advantages using network limited by single rigid alignment learn correlated with kinematics tool motion/interactions. Its loss function derived original transformation, re-projection error also pose comparison remote centre motion. proposed validated data da Vinci Si results indicate designed architecture extract relevant information estimate Unlike conventional approaches, estimations significantly simplifies problem RMIS context updating relationship done trained sequence images. This introduces potential creating approach capable accurately matrix according setup.